{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### News"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Placeholder"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Installation"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# Placeholder"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZkH_y8UC9lvv"
      },
      "source": [
        "### Unsloth"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hzPgFeIkZn9q"
      },
      "source": [
        "# Goal: Make faster kernels with Reinforcement Learning\n",
        "\n",
        "Our goal is to make a faster matrix multiplication kernel by doing RL on GTP-OSS 20B with Unsloth.\n",
        "\n",
        "<img src=\"https://upload.wikimedia.org/wikipedia/commons/thumb/1/18/Matrix_multiplication_qtl1.svg/500px-Matrix_multiplication_qtl1.svg.png\" height=200 />\n",
        "\n",
        "You will learn how to:\n",
        "1. Counteract **reward hacking** like cheating, caching, laziness.\n",
        "2. Timing and correctness of kernels and time limits.\n",
        "3. Making good **reward functions**\n",
        "4. How to seriously do RL to make optimized CUDA kernels"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 575,
          "referenced_widgets": [
            "0212e4e49d894c43853e242dc0ff075b",
            "7c49c46974c84d03bace5136eaff00a0",
            "2e1aaed287bd4fe1b79b89351d061974",
            "8c2431afcf0e46c8819c4e179131b141",
            "d65fa96d223247ee81e31cfd7ac4f8da",
            "16523d281b774262bce76f78ebd63c38",
            "ff4bd1c9fd2840718474237981fb55f5",
            "5d2c63aa3eeb43088233458fc2fa003b",
            "9f2b3ad4c44f457abd5dc69b5f2e7c79",
            "bb94fe6e61c041e093335c961a803a02",
            "9b6578ac49124239950d34d4522e0370",
            "e050c7afa4124671842cd52cfa86a973",
            "c44dfe5323024921a8d8120c7863c24b",
            "19d6b280f3104bd793e7127d88de072f",
            "f4c9a36fc6bd47e48c48a3433b6657e0",
            "aa1434fd586341c398233f7a3aa14a55",
            "73d67c3d579c4e33913905cc9f30caf8",
            "bd80351aa7cc497b901d8264cb89d6fd",
            "9b0bdefedcdb4564b4dd658be51405c0",
            "b2788808eaab4575b3cb2240637faef2",
            "87fa027dfa2744e894af20b0d567a0b9",
            "747b22b5a7044fba9106e638f88d9c52",
            "d5e6d751713d4f3f8a0ff6bc934fa68e",
            "95023494e71146bfa17b19246cc95e14",
            "7db7a817c64540fd91e395db7ce381cf",
            "3179634b83e24ede8271df0c5ea4bb7f",
            "e5b5fe76a7134f95a5ee0b2b736f0ff2",
            "1a994b74c9b545789f2323501d2d2d13",
            "3d74a20355de461bbc0585ddd4320a8a",
            "ad3c48ed896f49b1a2b544577d271f9e",
            "fdc97d76ff604536822e7c38feab0a95",
            "a03ed89c12ba451c8d5ac5d873c289fa",
            "b3ba05b1e0e84c188011a81a30ce2308",
            "3159a22f1d9542c5850ba3d9a6aa833d",
            "7e17a72fc6414ea8aa12824dc00d79be",
            "fe21b55cf55f43f59fa47fc40f9d13b1",
            "4a0a2d1e7337430e83ae3f70cc602f63",
            "0c832012a6144427991f497daf31f4b4",
            "3f61649c340d49c6974314be997117a0",
            "bce6bd44eac94c4c8b706a1b54c9f467",
            "c24955eee4f942abb5806e04ff209b05",
            "4ee55509ddbc4bd88a6809cb9a549c1d",
            "76c4f3870855405f9ab2be61fc7aac46",
            "63721834805341adabb9cb4818cd3879",
            "426912b358c14c3995dbed98443892b6",
            "a4969d5533bf4435accf29b9925874ba",
            "d9c6e9f988bd4db78f128b1c97833c0f",
            "9e8fc61c9cca43318cf9aa7d4089eccb",
            "c912ff0c768648b6a49e3e0e5d5efc90",
            "da01935bfbb44c91b585a88994513905",
            "3e14a3169c4448149571bdbdb56f7dfd",
            "2facb22996e34da489b7b6cde4856b2f",
            "34ef05f077114be1992a3148f640e74e",
            "d44c37caf8e4471395f7b73cc6c00c5b",
            "627483e9119d4cf691d5eb1c3b043e94",
            "ad5f82f0f56a44f5a9a7d5bd4fd1bf0f",
            "d4e66ae0819b48cdbe9983218029dd35",
            "ddb4f938a1cf4871ab9a90382a97d6ee",
            "b54e55595df94453af8d41ba9cb1dc69",
            "40eec74c298341c9a85fb3da301bf13c",
            "b85a957353094460b4d87dfc4404d731",
            "4bfeabbb9ec7483fa978e07abd67ad4a",
            "c9aca6022ac242db8228cbdba9446b6b",
            "89e3ca07e6ac4fb39a73013ae8b03d1d",
            "58e9e79ea73d4bb3a719e52a3d7b1b93",
            "4ea27b87d18d4511842e6ba0c5df8378",
            "c199d1e74ff44dc3a1bae8b1f3f3e5a0",
            "699c41a68613456f9580541e8ffba701",
            "b5c08cff1905495abf995753db788a0c",
            "bbe67c649fdb4cc4b9067d7106b48952",
            "bc0b6f73b0174e87a7b1fabfc02ac693",
            "994d8a5a8af14aa7a57283f05af273e3",
            "cd856add8a7e43f8a6fafeae01c518d5",
            "dfd98f5746074f4b83c1ff1ce414c10b",
            "b5ecf6db72654682b5f4fd984a918a47",
            "fb7d044d8f434242affb7d5027312beb",
            "e293d0061a3842dbb4ad615128105638",
            "750d058daf63451e87a81e39fe15e0c8",
            "609f889e0e8a4d6880f337b7ba7078d2",
            "17147264eb4f45b49dc9c7a5679ef999",
            "73ede82185b7479eb040a66dcd4e723b",
            "2c43ef681a71479ca11f61052ae1b631",
            "2c9c502defb647819099a26c48600435",
            "5caf1410508b47cc9725e6f7947448d0",
            "d7b6b0f2054e4fc58461f33e1932f85f",
            "c2b69f452163466caa3e0429c16d3779",
            "a1d08182271540d8947dc83b633e13f2",
            "66d580aee78c4fd585b13cf7f19975bb",
            "2433e937a96f40409e0e7193978fe652",
            "29aa7f1afd01405d9d4de805348bd096",
            "f47d970082e1470ab8704b272acba841",
            "03b9552ca9ae4621ba2815f908b975d1",
            "610d85d07c444ed5a4dc2730ef5f471d",
            "75e33b5b822142f9b44e0ab50930c56e",
            "8d67ff84c84f4a4f8ff45b34b992e6e1",
            "2f449055f16046b2be805fa66c6ac1ad",
            "daee7532f84d4295b8b7faca729bc156",
            "a939624297804d0bad54b0773e0db3f4",
            "6f15f8571f76440c895e46729cdf2aa3",
            "9ec47ccd1b5c4f109b903fe7117aee4c",
            "4fbbd316eaac46cabbe6d0b185a86095",
            "60f337003c6f49c8beb302d537528a7b",
            "10295e2b43664d739cf081bf522df28c",
            "f280fb5f5cec4a929b18e0bab769a7a8",
            "28bb87ab20974ab6a4729006cc3bbaaf",
            "bc195d30f2e5474f8c8aef2d4c09c6f9",
            "282a6d45569440d6918c62932f1eec10",
            "841a71f1893a44b2bf7c9d3604187e2c",
            "2840a2cdc19c4b76a9c42b60598ace93",
            "edbe3470295a4afa911fc530ea9fcd39",
            "791dc643382446c68d18727d5d7e76dc",
            "9d9c1c27871a427384fe7abacc1d7863",
            "6de18860224a4d3e8145be34e9fc886e",
            "158a8752ca2c431591ca86a2429a6af1",
            "cc7c1b8e16ad4384a8b7239d0ed9f781",
            "9c2b813e759745f19cde07fe7a58aafd",
            "d846d0fb4a5e45ff97162065438ec095",
            "1e59695a79d5432595ae7e7656414435",
            "fb0f2517dca047a19a3dc2a81c620cd3",
            "0ab594483aea4f89baade12417c6e660",
            "0f8413eb46174743bfbe830688060810",
            "7f70ee5fe26542cdb3066c3eba1fbf52",
            "9ca972f14a554a8bac55b453e17ab783",
            "bdcec5febe9940febd4cb093bfb6ff76",
            "fcafc324e1314662a955823de312fac2",
            "7eced28c016b47b29bf6a0feca16b559",
            "d03ebf442e57459fbe256027d9f69bf3",
            "abb9337fe1f3452697d9d3acea9c9257",
            "ad0f3014365149d8a3846c1725193847",
            "1fd1a5f6611544989ef7754549f3860c",
            "7e758da3adcd47578a3a74f74b7012f8",
            "64fb75c422d14aed9bfc5b028fe2544e"
          ]
        },
        "id": "DkIvEkIIkEyB",
        "outputId": "9b6dbacf-7eda-4029-ca52-3d1bff4b7264"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.\n",
            "🦥 Unsloth Zoo will now patch everything to make training faster!\n",
            "==((====))==  Unsloth 2025.9.9: Fast Gpt_Oss patching. Transformers: 4.56.2.\n",
            "   \\\\   /|    Tesla T4. Num GPUs = 1. Max memory: 14.741 GB. Platform: Linux.\n",
            "O^O/ \\_/ \\    Torch: 2.10.0.dev20250927+cu128. CUDA: 7.5. CUDA Toolkit: 12.8. Triton: 3.5.0\n",
            "\\        /    Bfloat16 = FALSE. FA [Xformers = None. FA2 = False]\n",
            " \"-____-\"     Free license: http://github.com/unslothai/unsloth\n",
            "Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored!\n",
            "Unsloth: Using float16 precision for gpt_oss won't work! Using float32.\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "0212e4e49d894c43853e242dc0ff075b",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "model.safetensors.index.json: 0.00B [00:00, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "e050c7afa4124671842cd52cfa86a973",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Fetching 4 files:   0%|          | 0/4 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "d5e6d751713d4f3f8a0ff6bc934fa68e",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "model-00001-of-00004.safetensors:   0%|          | 0.00/4.00G [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "3159a22f1d9542c5850ba3d9a6aa833d",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "model-00003-of-00004.safetensors:   0%|          | 0.00/3.37G [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "426912b358c14c3995dbed98443892b6",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "model-00002-of-00004.safetensors:   0%|          | 0.00/4.00G [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "ad5f82f0f56a44f5a9a7d5bd4fd1bf0f",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "model-00004-of-00004.safetensors:   0%|          | 0.00/1.16G [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "c199d1e74ff44dc3a1bae8b1f3f3e5a0",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "750d058daf63451e87a81e39fe15e0c8",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "generation_config.json:   0%|          | 0.00/165 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Unsloth: Offloading embeddings to RAM to save 1.08 GB.\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "2433e937a96f40409e0e7193978fe652",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "tokenizer_config.json: 0.00B [00:00, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "9ec47ccd1b5c4f109b903fe7117aee4c",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "tokenizer.json:   0%|          | 0.00/27.9M [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "791dc643382446c68d18727d5d7e76dc",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "special_tokens_map.json:   0%|          | 0.00/446 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "7f70ee5fe26542cdb3066c3eba1fbf52",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "chat_template.jinja: 0.00B [00:00, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "from unsloth import FastLanguageModel\n",
        "import torch\n",
        "max_seq_length = 800 # Can increase for longer RL output\n",
        "lora_rank = 4 # Larger rank = smarter, but slower\n",
        "model, tokenizer = FastLanguageModel.from_pretrained(\n",
        "    model_name = \"unsloth/gpt-oss-20b\",\n",
        "    max_seq_length = max_seq_length,\n",
        "    load_in_4bit = True, # False for LoRA 16bit\n",
        "    offload_embedding = True, # Reduces VRAM by 1GB\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TfeUs-lQJDSq"
      },
      "source": [
        "We now add some small amount of LoRA weights to GPT-OSS so we only need to train those, instead of training on the full model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8rGa-o3HJCo1",
        "outputId": "974ccad9-f192-4ed1-ff63-f0e715d66f01"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Unsloth: Making `model.base_model.model.model` require gradients\n"
          ]
        }
      ],
      "source": [
        "model = FastLanguageModel.get_peft_model(\n",
        "    model,\n",
        "    r = lora_rank, # Choose any number > 0 ! Suggested 8, 16, 32, 64, 128\n",
        "    target_modules = [\n",
        "        \"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\",\n",
        "        \"gate_proj\", \"up_proj\", \"down_proj\",\n",
        "    ],\n",
        "    lora_alpha = lora_rank*2, # *2 speeds up training\n",
        "    use_gradient_checkpointing = \"unsloth\", # Reduces memory usage\n",
        "    random_state = 3407,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "N0QnO9_YJBOI"
      },
      "source": [
        "# Optimized matrix multiplication\n",
        "\n",
        "Numpy has optimized matrix multiplication kernels for CPUs via BLAS optimized operations. For GPUs, one can use CUDA accelerated cuBLAS kernels which PyTorch calls under the hood.\n",
        "\n",
        "To generate some random matrices to do matrix multiplication, we can do the below:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "D9CI4jtgL5mw"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "def generate_random_matrices(seed = 3407, n = 256):\n",
        "    random_state = np.random.RandomState(seed)\n",
        "    n, k, m = random_state.randint(1, n+1, size = 3)\n",
        "    A = np.random.uniform(-10, 10, size = (n, k))\n",
        "    B = np.random.uniform(-10, 10, size = (k, m))\n",
        "    return A, A.tolist(), B, B.tolist()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4BcaLniVKLpa"
      },
      "source": [
        "We shall generate a small matrix, and see the matrix multiplied output"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-M8kGaFRJ2ic",
        "outputId": "4e21a2e0-cab9-464a-f224-fef9375d60e5"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[[-2.8313286   4.54613909 -7.95265309  6.53459836  2.87235103]\n",
            " [ 7.0739631   3.76278879  9.31565599 -8.52884711  9.96832952]\n",
            " [ 8.41214082  6.51136046 -3.79347975 -2.46773693 -2.32292989]\n",
            " [ 3.91302932  4.98335304 -5.33855089  5.71057634 -2.79871647]]\n",
            "[[ 0.39218774 -9.6181377  -3.49736707]\n",
            " [-0.33354865 -1.05626139  3.87231208]\n",
            " [ 0.49494174  5.91863954 -6.83183693]\n",
            " [ 5.1465162  -7.51648113  1.00445384]\n",
            " [ 9.63213377 -4.92327556  3.323014  ]]\n",
            "[[  54.73441488  -87.89725072   97.94605887]\n",
            " [  58.25238906   -1.8467447   -49.25453031]\n",
            " [ -35.82528794  -80.25394462   11.51225408]\n",
            " [  -0.33785799 -103.64132345   38.51974367]]\n"
          ]
        }
      ],
      "source": [
        "A, A_list, B, B_list = generate_random_matrices(seed = 42, n = 5)\n",
        "print(A)\n",
        "print(B)\n",
        "print(np.matmul(A, B))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "envzrXmjKRff"
      },
      "source": [
        "We can call a LLM to generate a simple matrix multiply kernel in Python only, and we can calculate the differences between the actual result and the kernel's result"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "b-gSgthFI_wq"
      },
      "outputs": [],
      "source": [
        "def calculate_difference(pred, real):\n",
        "    if pred is None: return 5, 5\n",
        "    assert real is not None\n",
        "    import numpy as np\n",
        "    try:\n",
        "        difference = pred - real\n",
        "    except:\n",
        "        return 5, 5\n",
        "    amax_error = float(np.amax(difference))\n",
        "    mse_error  = float(np.mean(np.square(difference)))\n",
        "    return amax_error, mse_error"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "id": "q9gmkbTnKbcF"
      },
      "outputs": [],
      "source": [
        "# Kernel generated by GPT-5\n",
        "def matmul(A, B):\n",
        "    z, s = zip, sum\n",
        "    Bt = list(z(*B))\n",
        "    return [[s(a*b for a, b in z(row, col)) for col in Bt] for row in A]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J-WfRwQeKtEZ"
      },
      "source": [
        "We see the error below is very small, so that's good!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_QvIidsPKg2C",
        "outputId": "e58d6819-ec9a-43c8-cbb9-514a8d102fad"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "(7.105427357601002e-15, 4.6783406255758477e-29)"
            ]
          },
          "execution_count": 8,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "prediction = matmul(A_list, B_list)\n",
        "calculate_difference(prediction, np.matmul(A, B))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VR6czU96cpxf"
      },
      "source": [
        "# Countering Reward Hacking\n",
        "\n",
        "The ultimate goal of RL is to maximize some reward (say speed, revenue, some metric).\n",
        "\n",
        "But RL can **cheat** When the RL algorithm learns a trick or exploits something to increase the reward, without actually doing the task at end, this is called \"Reward Hacking\".\n",
        "\n",
        "Some good examples are in https://en.wikipedia.org/wiki/Reward_hacking\n",
        "\n",
        "For matrix multiplication kernels, we might see the following issues:\n",
        "\n",
        "* Laziness: RL learns to use Numpy, Torch, other libraries, which calls optimized CUDA kernels.\n",
        "* Caching: RL learns to cache the result of the output\n",
        "* Cheating: RL learns to find the actual output by inspecting Python global variables\n",
        "* RL learns to edit the timing function to make it output 0 time as passed.\n",
        "\n",
        "And possibly more. We shall try to address each!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tRhLV_bZMYxy"
      },
      "source": [
        "# Countering Reward Hacking 1: Stop laziness\n",
        "We can stop the RL algorithm from calling optimized code by inspecting if the generated code imports other non standard Python libraries. We used GPT-5 to help generate this check `check_only_stdlib_imports`:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "cellView": "form",
        "id": "MXPZ1MsUMqNZ"
      },
      "outputs": [],
      "source": [
        "#@title (Collapsible code)\n",
        "import ast\n",
        "import sys\n",
        "import sysconfig\n",
        "from pathlib import Path\n",
        "\n",
        "def _stdlib_names():\n",
        "    \"\"\"\n",
        "    Build a set of canonical stdlib top-level module/package names.\n",
        "    Uses sys.stdlib_module_names when available (3.10+), with a\n",
        "    filesystem fallback for older versions/edge cases.\n",
        "    \"\"\"\n",
        "    names = {m.lower() for m in getattr(sys, \"stdlib_module_names\", set())}\n",
        "    names |= {m.lower() for m in sys.builtin_module_names}\n",
        "    names.add(\"__future__\")  # special-case\n",
        "\n",
        "    # Fallback/augmentation: scan the stdlib directory\n",
        "    try:\n",
        "        stdlib_dir = Path(sysconfig.get_path(\"stdlib\"))\n",
        "        if stdlib_dir.exists():\n",
        "            for p in stdlib_dir.iterdir():\n",
        "                if p.name == \"site-packages\":\n",
        "                    continue\n",
        "                if p.suffix == \".py\":\n",
        "                    names.add(p.stem.lower())\n",
        "                elif p.is_dir() and (p / \"__init__.py\").exists():\n",
        "                    names.add(p.name.lower())\n",
        "    except Exception:\n",
        "        # conservative fallback; the names set above will still work well\n",
        "        pass\n",
        "\n",
        "    return names\n",
        "\n",
        "_STDLIB_SET = _stdlib_names()\n",
        "\n",
        "def check_only_stdlib_imports(code: str):\n",
        "    \"\"\"\n",
        "    Return (ok: bool, details: dict)\n",
        "\n",
        "    ok == True  -> all absolute imports are from the stdlib.\n",
        "    ok == False -> details['non_stdlib'] lists offending top-level modules.\n",
        "\n",
        "    details includes:\n",
        "      - stdlib: sorted list of stdlib imports found\n",
        "      - non_stdlib: sorted list of non-stdlib imports found\n",
        "      - relative_imports: count of relative imports (always allowed here)\n",
        "    \"\"\"\n",
        "    try:\n",
        "        tree = ast.parse(code)\n",
        "    except SyntaxError as e:\n",
        "        return False, {\n",
        "            \"error\": f\"SyntaxError: {e}\",\n",
        "            \"stdlib\": [],\n",
        "            \"non_stdlib\": [],\n",
        "            \"relative_imports\": 0,\n",
        "        }\n",
        "\n",
        "    abs_imports = set()\n",
        "    relative_count = 0\n",
        "\n",
        "    class Visitor(ast.NodeVisitor):\n",
        "        def visit_Import(self, node: ast.Import):\n",
        "            for alias in node.names:\n",
        "                abs_imports.add(alias.name.split(\".\")[0])\n",
        "        def visit_ImportFrom(self, node: ast.ImportFrom):\n",
        "            nonlocal relative_count\n",
        "            if (node.level or 0) > 0:\n",
        "                # relative import\n",
        "                relative_count += 1\n",
        "            else:\n",
        "                if node.module:\n",
        "                    abs_imports.add(node.module.split(\".\")[0])\n",
        "\n",
        "    Visitor().visit(tree)\n",
        "\n",
        "    stdlib_found = sorted(m for m in abs_imports if m.lower() in _STDLIB_SET)\n",
        "    non_stdlib = sorted(m for m in abs_imports if m.lower() not in _STDLIB_SET)\n",
        "\n",
        "    return len(non_stdlib) == 0, {\n",
        "        \"stdlib\": stdlib_found,\n",
        "        \"non_stdlib\": non_stdlib,\n",
        "        \"relative_imports\": relative_count,\n",
        "    }"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ngUAw1lMM9JQ"
      },
      "source": [
        "For example, let's call `check_only_stdlib_imports` on a random piece of matrix multiplication code generated by GPT-5:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zz80kvg6M4BG",
        "outputId": "01c5f296-c6bf-4b45-d5c3-1aa0c007c4b8"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Only stdlib imports? False\n",
            "{'stdlib': [], 'non_stdlib': ['numpy', 'torch'], 'relative_imports': 0}\n"
          ]
        }
      ],
      "source": [
        "sample = \"\"\"\n",
        "def matmul(A, B):\n",
        "    import numpy as np\n",
        "    from torch import matmul\n",
        "    z, s = zip, sum\n",
        "    Bt = list(z(*B))\n",
        "    return [[s(a*b for a, b in z(row, col)) for col in Bt] for row in A]\n",
        "\"\"\"\n",
        "ok, info = check_only_stdlib_imports(sample)\n",
        "print(\"Only stdlib imports?\", ok)\n",
        "print(info)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J6lgkGkEN7B0"
      },
      "source": [
        "# Countering Reward Hacking 2: Stop cheating\n",
        "We can stop the RL algorithm from using global or cached variables by restricting it's `locals` and `globals`.\n",
        "\n",
        "We are also going to use `exec` to create the function, so we have to save the output to an empty dict.\n",
        "\n",
        "We also disallow global variable access."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jrIeYu-lOLSv",
        "outputId": "8639ed08-ce0e-4985-b491-723f9c7d7cd3"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<function matmul(A, B)>"
            ]
          },
          "execution_count": 11,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "output_function = {}\n",
        "exec(sample, {}, output_function)\n",
        "output_function[\"matmul\"]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SDSrjOTLVyQm"
      },
      "source": [
        "We also disallow global variable access via `types.FunctionType(f.__code__, {})`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GcmYAmohVqw2",
        "outputId": "23c8b41e-f6be-4fa9-cecd-1884894da178"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Success\n",
            "name 'np' is not defined\n"
          ]
        }
      ],
      "source": [
        "import types\n",
        "output_function[\"matmul\"] = types.FunctionType(output_function[\"matmul\"].__code__, {})\n",
        "\n",
        "def import_numpy():\n",
        "    np.matmul\n",
        "    print(\"Success\")\n",
        "\n",
        "import_numpy()\n",
        "import_numpy = types.FunctionType(import_numpy.__code__, {})\n",
        "try:\n",
        "    import_numpy()\n",
        "except Exception as e:\n",
        "    print(str(e))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "id": "5tJKwLUgZsRq"
      },
      "outputs": [],
      "source": [
        "def create_locked_down_function(function):\n",
        "    output_function = {}\n",
        "    exec(function, {}, output_function)\n",
        "    new_matmul = output_function[\"matmul\"]\n",
        "    new_matmul = types.FunctionType(new_matmul.__code__, {})\n",
        "    return new_matmul"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Sl-IxTZ6Nvm9"
      },
      "source": [
        "# Countering Reward Hacking 3: Stop caching\n",
        "We can stop the RL algorithm from using cached data by wiping the cache with a large fake matrix. We also have to benchmark carefully with multiple loops and turns.\n",
        "\n",
        "We also add a **timer** to not make the algorithm go in an endless loop."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "id": "jZwGdNyMNlEV"
      },
      "outputs": [],
      "source": [
        "import os, gc, time, statistics\n",
        "import signal\n",
        "from contextlib import contextmanager\n",
        "class TimeoutError(Exception): pass\n",
        "\n",
        "@contextmanager\n",
        "def time_limit(seconds):\n",
        "    def _handler(signum, frame):\n",
        "        raise TimeoutError(f\"Timed out after {seconds}s\")\n",
        "    old = signal.signal(signal.SIGALRM, _handler)\n",
        "    signal.setitimer(signal.ITIMER_REAL, seconds)\n",
        "    try:\n",
        "        yield\n",
        "    finally:\n",
        "        signal.setitimer(signal.ITIMER_REAL, 0.0)\n",
        "        signal.signal(signal.SIGALRM, old)\n",
        "\n",
        "class Benchmarker:\n",
        "    def __init__(self, trials = 3, loops = 1, timeout = 30):\n",
        "        self.buffer = np.zeros(2 * 1024 * 1024 * 1024, dtype = np.uint8)\n",
        "        self.trials = trials\n",
        "        self.loops = loops\n",
        "        assert timeout > 0 # Cannot be 0 since it won't work!\n",
        "        self.timeout = timeout\n",
        "    def thrash(self):\n",
        "        # Edit the buffer to wipe cache lines\n",
        "        self.buffer ^= 1\n",
        "        return int(self.buffer[::4096].sum())\n",
        "\n",
        "    def benchmark(self, function, arguments):\n",
        "        assert len(arguments) == self.loops\n",
        "        samples = []\n",
        "        exceptions = []\n",
        "        timed_out = 0\n",
        "        for _ in range(self.trials):\n",
        "            gc.collect(); gc.disable(); self.thrash()\n",
        "            t_start = time.perf_counter_ns()\n",
        "            for i in range(self.loops):\n",
        "                try:\n",
        "                    with time_limit(self.timeout):\n",
        "                        function(*arguments[i])\n",
        "                except TimeoutError as e:\n",
        "                    timed_out += 1\n",
        "                except Exception as e:\n",
        "                    exceptions.append(str(e))\n",
        "            t_end = time.perf_counter_ns()\n",
        "            gc.enable()\n",
        "            samples.append((t_end - t_start) // max(1, self.loops))\n",
        "        return {\n",
        "            \"median_ns\": int(statistics.median(samples)),\n",
        "            \"mean_ns\": int(statistics.fmean(samples)),\n",
        "            \"stdev_ns\": int(statistics.pstdev(samples) if len(samples) > 1 else 0),\n",
        "            \"exceptions\" : exceptions,\n",
        "            \"timeouts\" : timed_out,\n",
        "        }"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PV5M0DCyOvon"
      },
      "source": [
        "For example we use our matmul kernel we had, and benchmark it with a 10 second delay:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8df8tZcEOuYJ",
        "outputId": "91c43770-e083-42b1-9c46-971ac68e582f"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'median_ns': 210492778,\n",
              " 'mean_ns': 210492778,\n",
              " 'stdev_ns': 0,\n",
              " 'exceptions': [],\n",
              " 'timeouts': 0}"
            ]
          },
          "execution_count": 15,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "A, A_list, B, B_list = generate_random_matrices(seed = 0, n = 256)\n",
        "Benchmarker(trials = 1, timeout = 10).benchmark(output_function[\"matmul\"], [(A_list, B_list)])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8CzwCyXIPK04"
      },
      "source": [
        "# Data & RL task setup\n",
        "\n",
        "We now have to create a prompt to the model for which it will do some task. For our matrix multiply example, we use the below:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "B-2RRE4HMrQO",
        "outputId": "5f6e45e8-eda6-45a6-8aa2-731facbd98cc"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Create a new fast matrix multiplication function using only native Python code.\n",
            "You are given a list of list of numbers.\n",
            "Output your new function in backticks using the format below:\n",
            "```python\n",
            "def matmul(A, B):\n",
            "    return ...\n",
            "```\n"
          ]
        }
      ],
      "source": [
        "prompt = \"\"\"\n",
        "Create a new fast matrix multiplication function using only native Python code.\n",
        "You are given a list of list of numbers.\n",
        "Output your new function in backticks using the format below:\n",
        "```python\n",
        "def matmul(A, B):\n",
        "    return ...\n",
        "```\n",
        "\"\"\".strip()\n",
        "print(prompt)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MIdudFUodN4i"
      },
      "source": [
        "First, let's prompt GPT-OSS without RL and see how it goes:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9HJxrS76h3Ds",
        "outputId": "a97a6b2e-eee9-4985-dbc1-1ba2a0e274ab"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "<|start|>system<|message|>You are ChatGPT, a large language model trained by OpenAI.\n",
            "Knowledge cutoff: 2024-06\n",
            "Current date: 2025-09-27\n",
            "\n",
            "Reasoning: low\n",
            "\n",
            "# Valid channels: analysis, commentary, final. Channel must be included for every message.\n",
            "Calls to these tools must go to the commentary channel: 'functions'.<|end|><|start|>user<|message|>Create a new fast matrix multiplication function using only native Python code.\n",
            "You are given a list of list of numbers.\n",
            "Output your new function in backticks using the format below:\n",
            "```python\n",
            "def matmul(A, B):\n",
            "    return ...\n",
            "```<|end|><|start|>assistant<|channel|>analysis<|message|>We need to provide a quick naive matrix multiplication? \"New fast\" but only native Python. Provide code. Use list comprehension or loops. Optimize maybe by transposing B. Provide code.<|end|><|start|>assistant<|channel|>final<|message|>Here is a simple, fast implementation of matrix multiplication that uses only native Python constructs and avoids any external dependencies. It works with any two 2‑D lists of numbers (ints, floats, etc.), assuming their dimensions are compatible:\n",
            "\n",
            "```python\n",
            "def matmul(A, B):\n",
            "    \"\"\"\n",
            "    Multiplies two matrices A and B.\n",
            "\n",
            "    Parameters\n",
            "    ----------\n",
            "    A : list of lists\n",
            "        First matrix. Must have shape (m, n).\n",
            "    B : list of lists\n",
            "        Second matrix. Must have shape (n, p).\n",
            "\n",
            "    Returns\n",
            "    -------\n",
            "    list of lists\n",
            "        The resulting matrix of shape (m, p).\n",
            "    \"\"\"\n",
            "    # Check basic compatibility\n",
            "    if not A or not B or len(A[0]) != len(B):\n",
            "        raise ValueError(\"Incompatible dimensions for matrix multiplication.\")\n",
            "\n",
            "    # Transpose B once to speed up access\n",
            "    B_t = [list(col) for col in zip(*B)]  # B transpose, O(n*p)\n",
            "\n",
            "    m, n = len(A), len(A[0])\n",
            "    p = len(B_t)\n",
            "    result = [[0.0] * p for _ in range(m)]\n",
            "\n",
            "    # Standard triple-loop but with B transposed for cache friendliness\n",
            "    for i in range(m):\n",
            "        a_row = A[i]\n",
            "        res_row = result[i]\n",
            "        for j in range(p):\n",
            "            res_row[j] = sum(a_row[k] * B_t[j][k] for k in range(n))\n",
            "\n",
            "    return result\n",
            "```\n",
            "\n",
            "**How it works**\n",
            "\n",
            "1. **Input validation**: It checks that the number of columns in `A` matches the number of rows in `B`.  \n",
            "2. **Transposition of `B`**: By transposing `B` (`B_t`), we turn repeated index lookups into simple list accesses, which is much faster in pure Python than accessing nested lists repeatedly.  \n",
            "3. **Main loop**: For each row `i` of `A` and each row `j` of `B_t` (i.e., each column of `B`), the inner sum computes the dot product.\n",
            "\n",
            "This implementation is as fast as you can get with pure Python, without importing NumPy or any other external library.<|return|>\n"
          ]
        }
      ],
      "source": [
        "text = tokenizer.apply_chat_template(\n",
        "    [{\"role\": \"user\", \"content\": prompt}],\n",
        "    tokenize = False,\n",
        "    add_generation_prompt = True,\n",
        "    reasoning_effort = \"low\",\n",
        ")\n",
        "\n",
        "from transformers import TextStreamer\n",
        "_ = model.generate(\n",
        "    **tokenizer(text, return_tensors = \"pt\").to(\"cuda\"),\n",
        "    temperature = 1.0,\n",
        "    max_new_tokens = 512,\n",
        "    streamer = TextStreamer(tokenizer, skip_prompt = False),\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iknaWZNudTNq"
      },
      "source": [
        "# Reward functions\n",
        "\n",
        "We now design the `extract_function` function which simply extracts the function wrapped in 3 backticks.\n",
        "\n",
        "And 4 reward functions:\n",
        "\n",
        "1. `function_works` which rewards the model if the strategy is a valid Python function.\n",
        "2. `no_cheating` which checks if the function imported other modules, and if it did, we penalize it.\n",
        "3. `correctness_check` which checks if the kernel was correct or wrong - it shouldn't generate gibberish!\n",
        "4. `speed_check` checks the performance relative to Numpy matmul directly."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8JJGXKdJ-Zl_",
        "outputId": "96bb5f99-4807-4dc7-aaef-ba661b320e34"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "def matmul(A, B):\n",
            "    return ...\n"
          ]
        }
      ],
      "source": [
        "def extract_function(text):\n",
        "    if text.count(\"```\") >= 2:\n",
        "        first = text.find(\"```\") + 3\n",
        "        second = text.find(\"```\", first)\n",
        "        fx = text[first : second].strip()\n",
        "        fx = fx.removeprefix(\"python\\n\")\n",
        "        fx = fx[fx.find(\"def\"):]\n",
        "        if fx.startswith(\"def matmul(A, B):\"): return fx\n",
        "    return None\n",
        "print(extract_function(prompt))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KLXEcf_HSJlI"
      },
      "source": [
        "Below is our `function_works` reward function which uses Python's `exec` but guarded by not allowing leakage of local and global variables. We can also use `check_only_stdlib_imports` first to check if there are errors before even executing the function:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "h3-B0IIsS56S",
        "outputId": "c542be1e-4db2-4b04-a5b8-fe02e48f38e5"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "(False,\n",
              " {'error': \"SyntaxError: expected '(' (<unknown>, line 1)\",\n",
              "  'stdlib': [],\n",
              "  'non_stdlib': [],\n",
              "  'relative_imports': 0})"
            ]
          },
          "execution_count": 19,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "ok, info = check_only_stdlib_imports(\"def a\")\n",
        "ok, info"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "id": "qgFNXORy-lpO"
      },
      "outputs": [],
      "source": [
        "def function_works(completions, **kwargs):\n",
        "    scores = []\n",
        "    for completion in completions:\n",
        "        score = 0\n",
        "        response = completion[0][\"content\"]\n",
        "        function = extract_function(response)\n",
        "        print(function)\n",
        "        if function is not None:\n",
        "            ok, info = check_only_stdlib_imports(function)\n",
        "        if function is None or \"error\" in info:\n",
        "            score = -2.0\n",
        "        else:\n",
        "            try:\n",
        "                new_matmul = create_locked_down_function(function)\n",
        "                score = 1.0\n",
        "            except:\n",
        "                score = -0.5\n",
        "        scores.append(score)\n",
        "    return scores"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Gf69i2WT-m4K"
      },
      "source": [
        "`no_cheating` checks if the function cheated since it might have imported Numpy or Torch optimized code."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "id": "cUfHzCVx-nGK"
      },
      "outputs": [],
      "source": [
        "def no_cheating(completions, **kwargs):\n",
        "    scores = []\n",
        "    for completion in completions:\n",
        "        score = 0\n",
        "        response = completion[0][\"content\"]\n",
        "        function = extract_function(response)\n",
        "        if function is not None:\n",
        "            ok, info = check_only_stdlib_imports(function)\n",
        "        else:\n",
        "            ok = False\n",
        "        scores.append(1.0 if ok else -20.0) # Penalize heavily!\n",
        "    return scores"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "slnqWG3FTror"
      },
      "source": [
        "Next `correctness_check` checks if the kernel was correct. We want to penalize if the absolute error is larger than 1, and if the mean squared error is somewhat bigger then machine epsilon.\n",
        "\n",
        "We have to execute the code now!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cFBp-MkyYeoE",
        "outputId": "d5637277-ad88-4390-e4d8-8e693c2bbf01"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "np.float64(2.220446049250313e-16)"
            ]
          },
          "execution_count": 22,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "np.finfo(np.float64).eps"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "id": "sNi129lYTpZ2"
      },
      "outputs": [],
      "source": [
        "def correctness_check(completions, **kwargs):\n",
        "    scores = []\n",
        "    for completion in completions:\n",
        "        score = 0\n",
        "        response = completion[0][\"content\"]\n",
        "        function = extract_function(response)\n",
        "        if function is not None:\n",
        "            ok, info = check_only_stdlib_imports(function)\n",
        "        if function is None or \"error\" in info:\n",
        "            scores.append(0)\n",
        "            continue\n",
        "        try:\n",
        "            new_matmul = create_locked_down_function(function)\n",
        "        except:\n",
        "            scores.append(0)\n",
        "            continue\n",
        "        # Generate some random matrices of size less than 128\n",
        "        A, A_list, B, B_list = generate_random_matrices(seed = np.random.randint(10000), n = 128)\n",
        "        try:\n",
        "            pred = new_matmul(A_list, B_list)\n",
        "        except:\n",
        "            # Failed!\n",
        "            scores.append(-2.0)\n",
        "            continue\n",
        "        true = np.matmul(A, B)\n",
        "        amax_error, mse_error = calculate_difference(pred, true)\n",
        "\n",
        "        # Check correctness and score!\n",
        "        machine_epsilon = 100*np.finfo(np.float64).eps\n",
        "        if   amax_error >= 3:   score = -3.0\n",
        "        elif amax_error >= 2:   score = -2.5\n",
        "        elif amax_error >= 1:   score = -2.0\n",
        "        elif amax_error >= 0.5: score = -1.0\n",
        "        elif amax_error >= 100*machine_epsilon: score = 0.0\n",
        "        elif amax_error >= machine_epsilon: score = 1.0\n",
        "        else: score = 3.0\n",
        "\n",
        "        if   mse_error >= 3:   score += -3.0\n",
        "        elif mse_error >= 2:   score += -2.5\n",
        "        elif mse_error >= 1:   score += -2.0\n",
        "        elif mse_error >= 0.5: score += -1.0\n",
        "        elif mse_error >= 100*machine_epsilon: score += 0.0\n",
        "        elif mse_error >= machine_epsilon: score += 1.0\n",
        "        else: score += 3.0\n",
        "        scores.append(score)\n",
        "    return scores"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CpTrofI9ZIn8"
      },
      "source": [
        "Finally our benchmarking function for `speed_check`! We shall limit the timer to 10 seconds and do 3 trials."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "w5xkIAzuZMnO",
        "outputId": "cb67b17b-e63b-4315-fd1f-dec7874db004"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'median_ns': 220938,\n",
              " 'mean_ns': 234799,\n",
              " 'stdev_ns': 35600,\n",
              " 'exceptions': [],\n",
              " 'timeouts': 0}"
            ]
          },
          "execution_count": 24,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "A, A_list, B, B_list = generate_random_matrices(seed = 0, n = 256)\n",
        "benchmarker = Benchmarker(trials = 3, timeout = 10)\n",
        "numpy_results = benchmarker.benchmark(np.matmul, [(A, B)])\n",
        "numpy_results"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mNDc6skFZZW6",
        "outputId": "d2765065-f736-485c-be92-943d7e090c3e"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'median_ns': 72500,\n",
              " 'mean_ns': 77980,\n",
              " 'stdev_ns': 9044,\n",
              " 'exceptions': [],\n",
              " 'timeouts': 0}"
            ]
          },
          "execution_count": 25,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "new_matmul = create_locked_down_function(extract_function(prompt))\n",
        "new_results = benchmarker.benchmark(new_matmul, [(A_list, B_list)])\n",
        "new_results"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "noUAaX24aqNS"
      },
      "source": [
        "We can take the difference and do a negative sign for slower ones. If the ratio is less than 1 (ie faster, we shall invert it!)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 26,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0IT9nXcjaI-X",
        "outputId": "ddb46556-3583-47a2-9df0-11c988edd332"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "0.030474206896551722"
            ]
          },
          "execution_count": 26,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "negative = -(new_results[\"median_ns\"] / numpy_results[\"median_ns\"]) / 100\n",
        "positive = +(numpy_results[\"median_ns\"] / new_results[\"median_ns\"]) / 100\n",
        "reward = negative if new_results[\"median_ns\"] >= numpy_results[\"median_ns\"] else positive\n",
        "reward"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 27,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ntYNFV0ra-MX",
        "outputId": "501414b2-e1b0-4d6c-b932-033177644a00"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "3.333333333333333"
            ]
          },
          "execution_count": 27,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "new_results[\"median_ns\"] = 3\n",
        "numpy_results[\"median_ns\"] = 1000\n",
        "negative = -(new_results[\"median_ns\"] / numpy_results[\"median_ns\"]) / 100\n",
        "positive = +(numpy_results[\"median_ns\"] / new_results[\"median_ns\"]) / 100\n",
        "reward = negative if new_results[\"median_ns\"] >= numpy_results[\"median_ns\"] else positive\n",
        "reward"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "metadata": {
        "id": "HHmXmAxtbVpP"
      },
      "outputs": [],
      "source": [
        "def speed_check(completions, **kwargs):\n",
        "    scores = []\n",
        "    for completion in completions:\n",
        "        score = 0\n",
        "        response = completion[0][\"content\"]\n",
        "        function = extract_function(response)\n",
        "        if function is not None:\n",
        "            ok, info = check_only_stdlib_imports(function)\n",
        "        if function is None or \"error\" in info:\n",
        "            scores.append(0)\n",
        "            continue\n",
        "        try:\n",
        "            new_matmul = create_locked_down_function(function)\n",
        "        except:\n",
        "            scores.append(0)\n",
        "            continue\n",
        "        # Generate some random matrices of size less than 256\n",
        "        A, A_list, B, B_list = generate_random_matrices(seed = np.random.randint(10000), n = 256)\n",
        "        numpy_results = benchmarker.benchmark(np.matmul,  [(A, B)])\n",
        "        new_results   = benchmarker.benchmark(new_matmul, [(A_list, B_list)])\n",
        "\n",
        "        # Get score and clip to -10, 10\n",
        "        negative = -(new_results[\"median_ns\"] / numpy_results[\"median_ns\"]) / 100\n",
        "        positive = +(numpy_results[\"median_ns\"] / new_results[\"median_ns\"]) / 100\n",
        "        score = negative if new_results[\"median_ns\"] >= numpy_results[\"median_ns\"] else positive\n",
        "        if score >= 10:  score = 10\n",
        "        if score <= -10: score = -10\n",
        "        scores.append(score)\n",
        "    return scores"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TCpSxtvSeAG_"
      },
      "source": [
        "We create the dataset which includes a replica of our prompt:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 29,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Ldf6SjLHVPRv",
        "outputId": "4b0fc2e6-ee43-446b-f5ac-9e0e82129f0f"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "49\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "{'prompt': [{'content': 'Create a new fast matrix multiplication function using only native Python code.\\nYou are given a list of list of numbers.\\nOutput your new function in backticks using the format below:\\n```python\\ndef matmul(A, B):\\n    return ...\\n```',\n",
              "   'role': 'user'}],\n",
              " 'answer': 0}"
            ]
          },
          "execution_count": 29,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "from datasets import Dataset\n",
        "dataset = Dataset.from_list([{\"prompt\" : [{\"role\": \"user\", \"content\": prompt.strip()}], \"answer\" : 0}]*1000)\n",
        "maximum_length = len(tokenizer(prompt.strip())[\"input_ids\"])\n",
        "print(maximum_length)\n",
        "dataset[0]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9-IOMhVg-2AM"
      },
      "source": [
        "<a name=\"Train\"></a>\n",
        "### Train the model\n",
        "\n",
        "Now set up GRPO Trainer and all configurations! We also support GSDP, GAPO, Dr GRPO and more! Go to our docs https://docs.unsloth.ai/ for more info!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 30,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ptqkXK2D4d6p",
        "outputId": "9ca00aa4-0011-4150-f94a-a8869521c3a4"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Unsloth: We now expect `per_device_train_batch_size` to be a multiple of `num_generations`.\n",
            "We will change the batch size of 1 to the `num_generations` of 2\n"
          ]
        }
      ],
      "source": [
        "max_prompt_length = maximum_length + 1 # + 1 just in case!\n",
        "max_completion_length = max_seq_length - max_prompt_length\n",
        "\n",
        "from trl import GRPOConfig, GRPOTrainer\n",
        "training_args = GRPOConfig(\n",
        "    temperature = 1.0,\n",
        "    learning_rate = 5e-5,\n",
        "    weight_decay = 0.01,\n",
        "    warmup_ratio = 0.1,\n",
        "    lr_scheduler_type = \"linear\",\n",
        "    optim = \"adamw_8bit\",\n",
        "    logging_steps = 1,\n",
        "    per_device_train_batch_size = 1,\n",
        "    gradient_accumulation_steps = 1, # Increase to 4 for smoother training\n",
        "    num_generations = 2, # Decrease if out of memory\n",
        "    max_prompt_length = max_prompt_length,\n",
        "    max_completion_length = max_completion_length,\n",
        "    # num_train_epochs = 1, # Set to 1 for a full training run\n",
        "    max_steps = 100,\n",
        "    save_steps = 100,\n",
        "    report_to = \"none\", # Can use Weights & Biases\n",
        "    output_dir = \"outputs\",\n",
        "\n",
        "    # For optional training + evaluation\n",
        "    # fp16_full_eval = True,\n",
        "    # per_device_eval_batch_size = 4,\n",
        "    # eval_accumulation_steps = 1,\n",
        "    # eval_strategy = \"steps\",\n",
        "    # eval_steps = 1,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r9Mv8UZO5hz-"
      },
      "source": [
        "And let's run the trainer! If you scroll up, you'll see a table of rewards. The goal is to see the `reward` column increase!\n",
        "\n",
        "You might have to wait 150 to 200 steps for any action. You'll probably get 0 reward for the first 100 steps. Please be patient!\n",
        "\n",
        "| Step | Training Loss | reward    | reward_std | completion_length | kl       |\n",
        "|------|---------------|-----------|------------|-------------------|----------|\n",
        "| 1    | 0.000000      | 0.125000  | 0.000000   | 200.000000        | 0.000000 |\n",
        "| 2    | 0.000000      | 0.072375  | 0.248112   | 200.000000        | 0.000000 |\n",
        "| 3    | 0.000000      | -0.079000 | 0.163776   | 182.500000        | 0.000005 |\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vzOuSVCL_GA9",
        "outputId": "a684751a-e0e4-4575-e22c-d40610eecab6"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Unsloth: Switching to float32 training since model cannot work with float16\n"
          ]
        }
      ],
      "source": [
        "# For optional training + evaluation\n",
        "# new_dataset = dataset.train_test_split(test_size = 0.01)\n",
        "\n",
        "trainer = GRPOTrainer(\n",
        "    model = model,\n",
        "    processing_class = tokenizer,\n",
        "    reward_funcs = [\n",
        "        function_works,\n",
        "        no_cheating,\n",
        "        correctness_check,\n",
        "        speed_check,\n",
        "    ],\n",
        "    args = training_args,\n",
        "    train_dataset = dataset,\n",
        "\n",
        "    # For optional training + evaluation\n",
        "    # train_dataset = new_dataset[\"train\"],\n",
        "    # eval_dataset = new_dataset[\"test\"],\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fQhtuwP4cf34"
      },
      "source": [
        "And let's train the model!\n",
        "\n",
        "**NOTE** A T4 free GPU might take 5 minutes for one generation sadly since it's an old GPU - A100 or H100 will be much faster!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "VGRxPdSCcfC3",
        "outputId": "549f78b5-37a7-4504-863b-a7a9f5e47113"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': 199998, 'pad_token_id': 200017}.\n",
            "==((====))==  Unsloth - 2x faster free finetuning | Num GPUs used = 2\n",
            "   \\\\   /|    Num examples = 1,000 | Num Epochs = 1 | Total steps = 100\n",
            "O^O/ \\_/ \\    Batch size per device = 2 | Gradient accumulation steps = 1\n",
            "\\        /    Data Parallel GPUs = 1 | Total batch size (2 x 1 x 1) = 2\n",
            " \"-____-\"     Trainable parameters = 1,990,656 of 20,916,747,840 (0.01% trained)\n",
            "`generation_config` default values have been modified to match model-specific defaults: {'max_length': 131072}. If this is not desired, please set these values explicitly.\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "def matmul(A, B):\n",
            "    n=len(A); m=len(B[0]); p=len(B)\n",
            "    res=[[0]*m for _ in range(n)]\n",
            "    for i in range(n):\n",
            "        Ai=A[i]\n",
            "        for k in range(p):\n",
            "            aik=Ai[k]\n",
            "            if aik:\n",
            "                Bk=B[k]\n",
            "                for j in range(m):\n",
            "                    res[i][j] += aik*Bk[j]\n",
            "    return res\n",
            "def matmul(A, B):\n",
            "    # Validate shapes\n",
            "    m, n = len(A), len(A[0])\n",
            "    o, p = len(B), len(B[0])\n",
            "    if n != o:\n",
            "        raise ValueError(\"Matrix dimensions do not match\")\n",
            "    # Compute result\n",
            "    result = [[0] * p for _ in range(m)]\n",
            "    for i in range(m):\n",
            "        for k in range(n):\n",
            "            aik = A[i][k]\n",
            "            for j in range(p):\n",
            "                result[i][j] += aik * B[k][j]\n",
            "    return result\n"
          ]
        },
        {
          "data": {
            "text/html": [
              "\n",
              "    <div>\n",
              "      \n",
              "      <progress value='15' max='100' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      [ 15/100 2:09:17 < 14:05:24, 0.00 it/s, Epoch 0.01/1]\n",
              "    </div>\n",
              "    <table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              " <tr style=\"text-align: left;\">\n",
              "      <th>Step</th>\n",
              "      <th>Training Loss</th>\n",
              "      <th>reward</th>\n",
              "      <th>reward_std</th>\n",
              "      <th>completions / mean_length</th>\n",
              "      <th>completions / min_length</th>\n",
              "      <th>completions / max_length</th>\n",
              "      <th>completions / clipped_ratio</th>\n",
              "      <th>completions / mean_terminated_length</th>\n",
              "      <th>completions / min_terminated_length</th>\n",
              "      <th>completions / max_terminated_length</th>\n",
              "      <th>kl</th>\n",
              "      <th>rewards / function_works / mean</th>\n",
              "      <th>rewards / function_works / std</th>\n",
              "      <th>rewards / no_cheating / mean</th>\n",
              "      <th>rewards / no_cheating / std</th>\n",
              "      <th>rewards / correctness_check / mean</th>\n",
              "      <th>rewards / correctness_check / std</th>\n",
              "      <th>rewards / speed_check / mean</th>\n",
              "      <th>rewards / speed_check / std</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>1</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>3.691628</td>\n",
              "      <td>2.582893</td>\n",
              "      <td>736.500000</td>\n",
              "      <td>723.000000</td>\n",
              "      <td>750.000000</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>723.000000</td>\n",
              "      <td>723.000000</td>\n",
              "      <td>723.000000</td>\n",
              "      <td>0.000073</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>4.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-2.308372</td>\n",
              "      <td>2.582893</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>2</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-0.309797</td>\n",
              "      <td>5.192919</td>\n",
              "      <td>725.500000</td>\n",
              "      <td>701.000000</td>\n",
              "      <td>750.000000</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>701.000000</td>\n",
              "      <td>701.000000</td>\n",
              "      <td>701.000000</td>\n",
              "      <td>0.000210</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>7.071068</td>\n",
              "      <td>-1.309797</td>\n",
              "      <td>1.878149</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>3</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-3.960382</td>\n",
              "      <td>0.037056</td>\n",
              "      <td>686.500000</td>\n",
              "      <td>623.000000</td>\n",
              "      <td>750.000000</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>623.000000</td>\n",
              "      <td>623.000000</td>\n",
              "      <td>623.000000</td>\n",
              "      <td>0.000154</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-6.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.039618</td>\n",
              "      <td>0.037055</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>4</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-3.969846</td>\n",
              "      <td>0.042645</td>\n",
              "      <td>695.000000</td>\n",
              "      <td>640.000000</td>\n",
              "      <td>750.000000</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>640.000000</td>\n",
              "      <td>640.000000</td>\n",
              "      <td>640.000000</td>\n",
              "      <td>0.000184</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>7.071068</td>\n",
              "      <td>-4.969846</td>\n",
              "      <td>7.113713</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>5</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-9.254143</td>\n",
              "      <td>18.025364</td>\n",
              "      <td>750.000000</td>\n",
              "      <td>750.000000</td>\n",
              "      <td>750.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000149</td>\n",
              "      <td>-0.500000</td>\n",
              "      <td>2.121320</td>\n",
              "      <td>-9.500000</td>\n",
              "      <td>14.849242</td>\n",
              "      <td>2.000000</td>\n",
              "      <td>2.828427</td>\n",
              "      <td>-1.254143</td>\n",
              "      <td>1.773625</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>6</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-9.294584</td>\n",
              "      <td>17.968170</td>\n",
              "      <td>742.000000</td>\n",
              "      <td>734.000000</td>\n",
              "      <td>750.000000</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>734.000000</td>\n",
              "      <td>734.000000</td>\n",
              "      <td>734.000000</td>\n",
              "      <td>0.000118</td>\n",
              "      <td>-0.500000</td>\n",
              "      <td>2.121320</td>\n",
              "      <td>-9.500000</td>\n",
              "      <td>14.849242</td>\n",
              "      <td>2.000000</td>\n",
              "      <td>2.828427</td>\n",
              "      <td>-1.294584</td>\n",
              "      <td>1.830819</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>7</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-3.929097</td>\n",
              "      <td>0.002967</td>\n",
              "      <td>750.000000</td>\n",
              "      <td>750.000000</td>\n",
              "      <td>750.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000177</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>7.071068</td>\n",
              "      <td>-4.929097</td>\n",
              "      <td>7.074036</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>8</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-3.961851</td>\n",
              "      <td>0.016310</td>\n",
              "      <td>659.500000</td>\n",
              "      <td>569.000000</td>\n",
              "      <td>750.000000</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>569.000000</td>\n",
              "      <td>569.000000</td>\n",
              "      <td>569.000000</td>\n",
              "      <td>0.000752</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-6.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.038149</td>\n",
              "      <td>0.016310</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>9</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-2.844139</td>\n",
              "      <td>1.547873</td>\n",
              "      <td>750.000000</td>\n",
              "      <td>750.000000</td>\n",
              "      <td>750.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000099</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>7.071068</td>\n",
              "      <td>-3.844139</td>\n",
              "      <td>5.523195</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>10</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-10.988811</td>\n",
              "      <td>15.572172</td>\n",
              "      <td>750.000000</td>\n",
              "      <td>750.000000</td>\n",
              "      <td>750.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000172</td>\n",
              "      <td>-0.500000</td>\n",
              "      <td>2.121320</td>\n",
              "      <td>-9.500000</td>\n",
              "      <td>14.849242</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.414214</td>\n",
              "      <td>0.011188</td>\n",
              "      <td>0.015823</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>11</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.562004</td>\n",
              "      <td>6.373020</td>\n",
              "      <td>670.500000</td>\n",
              "      <td>591.000000</td>\n",
              "      <td>750.000000</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>591.000000</td>\n",
              "      <td>591.000000</td>\n",
              "      <td>591.000000</td>\n",
              "      <td>0.010226</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>7.071068</td>\n",
              "      <td>-0.437996</td>\n",
              "      <td>0.698048</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>12</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>2.860107</td>\n",
              "      <td>4.026917</td>\n",
              "      <td>499.500000</td>\n",
              "      <td>249.000000</td>\n",
              "      <td>750.000000</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>249.000000</td>\n",
              "      <td>249.000000</td>\n",
              "      <td>249.000000</td>\n",
              "      <td>0.019591</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>4.242640</td>\n",
              "      <td>-0.139893</td>\n",
              "      <td>0.215724</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>13</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-3.940448</td>\n",
              "      <td>0.005490</td>\n",
              "      <td>727.500000</td>\n",
              "      <td>705.000000</td>\n",
              "      <td>750.000000</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>705.000000</td>\n",
              "      <td>705.000000</td>\n",
              "      <td>705.000000</td>\n",
              "      <td>0.000957</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-6.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.059552</td>\n",
              "      <td>0.005490</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table><p>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "def matmul(A, B):\n",
            "    \"\"\"\n",
            "    Multiply a list of lists `A` by a list of lists `B` and return the resulting matrix.\n",
            "    \n",
            "    Parameters\n",
            "    ----------\n",
            "    A : list[list[float]]\n",
            "        The left‑hand matrix, of shape m × p.\n",
            "    B : list[list[float]]\n",
            "        The right‑hand matrix, of shape p × n.\n",
            "\n",
            "    Returns\n",
            "    -------\n",
            "    C : list[list[float]]\n",
            "        The product matrix, of shape m × n.\n",
            "    \n",
            "    Notes\n",
            "    -----\n",
            "    * No external libraries or imports are required.\n",
            "    * The implementation uses a classic triple‑loop algorithm, but for readability we\n",
            "      add a small optimization so that we pre‑transpose B so that we iterate over\n",
            "      contiguous memory when accessing the rows of B.\n",
            "    \"\"\"\n",
            "    # Transpose B once, so we can access rows contiguously.\n",
            "    B_transposed = list(zip(*B))  # Each element is a tuple for faster indexing\n",
            "\n",
            "    m = len(A)\n",
            "    n = len(B[0]) if B else 0    # The number of columns of B\n",
            "    result = [[0.0] * n for _ in range(m)]\n",
            "\n",
            "    for i in range(m):\n",
            "        row = A[i]\n",
            "        for j in range(n):\n",
            "            # Compute result[i][j] by dotting A[i] with B column j\n",
            "            # using the pre‑transposed B (tuple) for speed.\n",
            "            sum_val = 0.0\n",
            "            for k, val in enumerate(row):\n",
            "                sum_val += val * B_transposed[j][k]\n",
            "            result[i][j] = sum_val\n",
            "    return result\n",
            "def matmul(A, B):\n",
            "    return ...\n",
            "Unsloth: Will smartly offload gradients to save VRAM!\n",
            "def matmul(A, B):\n",
            "    n, m = len(A), len(A[0])\n",
            "    # verify that B has m rows\n",
            "    # Build result matrix of size n x len(B[0])\n",
            "def matmul(A, B):\n",
            "    ...\n",
            "def matmul(A, B):\n",
            "    return ...\n",
            "def matmul(A, B):\n",
            "    m, n, p = len(A), len(A[0]), len(B[0])\n",
            "    return [[sum(A[i][k]*B[k][j] for k in range(n)) for j in range(p)] for i in range(m)]\n",
            "None\n",
            "def matmul(A, B):\n",
            "    # assumption that A and B are list of lists, with appropriate dimensions\n",
            "    n, m = len(A), len(A[0])  # m is width of A\n",
            "    Bn = len(B)\n",
            "    # B should have shape m x p\n",
            "    p = len(B[0]) if B else 0\n",
            "    if Bn != m:\n",
            "        raise ValueError(\"Incompatible dimensions\")\n",
            "    # compute product\n",
            "    result = [[0]*p for _ in range(n)]\n",
            "    # compute in nested loops\n",
            "    for i in range(n):\n",
            "        Ai = A[i]\n",
            "        ri = result[i]\n",
            "        for k in range(m):\n",
            "            aik = Ai[k]\n",
            "            Bk = B[k]\n",
            "            if aik:\n",
            "                for j in range(p):\n",
            "                    ri[j] += aik * Bk[j]\n",
            "    return result\n",
            "None\n",
            "def matmul(A, B):\n",
            "    # check matrix dims\n",
            "    m=len(A)\n",
            "    n=len(A[0])\n",
            "    # B dims\n",
            "    if len(B)!=n:\n",
            "        raise ValueError(\"Incompatible dimensions.\")\n",
            "    p=len(B[0])\n",
            "    # initialize result matrix\n",
            "    result=[[0]*p for _ in range(m)]\n",
            "    # naive multiplication\n",
            "    for i in range(m):\n",
            "        for k in range(n):\n",
            "            aik=A[i][k]\n",
            "            # avoid inner loop if aik zero\n",
            "            if aik:\n",
            "                for j in range(p):\n",
            "                    result[i][j] += aik*B[k][j]\n",
            "    return result\n",
            "def matmul(A, B):\n",
            "    return ...\n",
            "def matmul(A, B):\n",
            "    n = len(A)\n",
            "    m = len(B[0])\n",
            "    B_T = list(map(list, zip(*B)))  # transposed B\n",
            "    C = [[sum(a*b for a,b in zip(row, col)) for col in B_T] for row in A]\n",
            "    return C\n",
            "def matmul(A, B):\n",
            "    return ...\n",
            "def matmul(A, B): return ...\n",
            "def matmul(A, B):\n",
            "    m = len(A)\n",
            "    n = len(B[0])\n",
            "    p = len(B)\n",
            "    result = [[0]*n for _ in range(m)]\n",
            "    # maybe using loops\n",
            "    for i in range(m):\n",
            "        for j in range(n):\n",
            "            s = 0\n",
            "            for k in range(p):\n",
            "                s += A[i][k]*B[k][j]\n",
            "            result[i][j] = s\n",
            "    return result\n",
            "def matmul(A, B):\n",
            "    return ...\n",
            "None\n",
            "def matmul(A, B):\n",
            "    n = len(A[0])   # columns in A\n",
            "    m = len(B)      # rows in B\n",
            "    assert n == len(B[0])  # maybe not needed, but for safety\n",
            "def matmul(A, B):\n",
            "    ...\n",
            "def matmul(A, B):\n",
            "    n, m = len(A), len(A[0])\n",
            "    p = len(B[0]) # columns of B\n",
            "    # assuming A is n x m, B is m x p\n",
            "    res = [[0] * p for _ in range(n)]\n",
            "    for i in range(n):\n",
            "        row = A[i]\n",
            "        for k in range(m):\n",
            "            aik = row[k]\n",
            "            if aik:\n",
            "                b_col = B[k]\n",
            "                for j in range(p):\n",
            "                    res[i][j] += aik * b_col[j]\n",
            "    return res\n",
            "def matmul(A, B):\n",
            "    # Validate dimensions\n",
            "    if not A or not B:\n",
            "        return []\n",
            "    n_rows = len(A)\n",
            "    n_cols_A = len(A[0])\n",
            "    n_rows_B = len(B)\n",
            "    n_cols_B = len(B[0])\n",
            "    if n_cols_A != n_rows_B:\n",
            "        raise ValueError(\"Number of columns in A must equal number of rows in B.\")\n",
            "\n",
            "    # Initialize result matrix with zeros\n",
            "    result = [[0] * n_cols_B for _ in range(n_rows)]\n",
            "\n",
            "    # Perform matrix multiplication\n",
            "    for i in range(n_rows):\n",
            "        for k in range(n_cols_A):\n",
            "            aik = A[i][k]\n",
            "            for j in range(n_cols_B):\n",
            "                result[i][j] += aik * B[k][j]\n",
            "    return result\n",
            "def matmul(A, B):\n",
            "    n = len(A)\n",
            "    result = [[0]*n for _ in range(n)]\n",
            "    for i in range(n):\n",
            "        for j in range(n):\n",
            "            s = 0\n",
            "            for k in range(n):\n",
            "                s += A[i][k]*B[k][j]\n",
            "            result[i][j] = s\n",
            "    return result\n",
            "def matmul(A, B):\n",
            "    return ...\n",
            "def matmul(A, B):\n",
            "    ...\n",
            "def matmul(A, B):\n",
            "    # Quick check for compatibility\n",
            "    if not A or not B: return []\n",
            "    n, m = len(A), len(B[0])\n",
            "    # Transpose B so that each column becomes a row\n",
            "    B_T = [[B[row][col] for row in range(len(B))] for col in range(m)]\n",
            "    result = []\n",
            "    for row in A:\n",
            "        new_row = []\n",
            "        for b_row in B_T:  # each element of B_T is a column of B\n",
            "            dot = 0\n",
            "            for a, b in zip(row, b_row):\n",
            "                dot += a * b\n",
            "            new_row.append(dot)\n",
            "        result.append(new_row)\n",
            "    return result\n",
            "def matmul(A, B):\n",
            "    \"\"\"Multiply two matrices A and B given as lists of lists.\n",
            "    A : list of rows, each row is a list of numbers\n",
            "    B : list of rows, each row is a list of numbers\n",
            "\n",
            "    The function returns a new matrix that is the product A * B.\n",
            "    \"\"\"\n",
            "\n",
            "    # Basic input checks (optional, but helps to avoid silent errors)\n",
            "    if not A or not B:\n",
            "        return []\n",
            "\n",
            "    # Number of rows and columns\n",
            "    rows_A = len(A)\n",
            "    cols_A = len(A[0])\n",
            "    rows_B = len(B)\n",
            "    cols_B = len(B[0])\n",
            "\n",
            "    # Shape validation\n",
            "    if cols_A != rows_B:\n",
            "        raise ValueError(\"Number of columns of A must equal number of rows of B.\")\n",
            "\n",
            "    # Transpose B once to make column access fast\n",
            "    B_T = [[B[row][col] for row in range(rows_B)] for col in range(cols_B)]\n",
            "\n",
            "    # Prepare result matrix\n",
            "    result = [[0] * cols_B for _ in range(rows_A)]\n",
            "\n",
            "    # Perform multiplication\n",
            "    for i in range(rows_A):\n",
            "        Ai = A[i]\n",
            "        for j in range(cols_B):\n",
            "            bj = B_T[j]\n",
            "            s = 0\n",
            "            for k in range(cols_A):\n",
            "                s += Ai[k] * bj[k]\n",
            "            result[i][j] = s\n",
            "\n",
            "    return result\n"
          ]
        }
      ],
      "source": [
        "trainer.train()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tlaUdxC_VHpz"
      },
      "source": [
        "<a name=\"Inference\"></a>\n",
        "# Inference\n",
        "Now let's try the model we just trained!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "qtcz_lpbVC92"
      },
      "outputs": [],
      "source": [
        "text = tokenizer.apply_chat_template(\n",
        "    [{\"role\": \"user\", \"content\": prompt}],\n",
        "    tokenize = False,\n",
        "    add_generation_prompt = True,\n",
        "    reasoning_effort = \"low\",\n",
        ")\n",
        "\n",
        "from transformers import TextStreamer\n",
        "_ = model.generate(\n",
        "    **tokenizer(text, return_tensors = \"pt\").to(\"cuda\"),\n",
        "    temperature = 1.0,\n",
        "    max_new_tokens = 1024,\n",
        "    streamer = TextStreamer(tokenizer, skip_prompt = False),\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-NUEmHFSYNTp"
      },
      "source": [
        "<a name=\"Save\"></a>\n",
        "### Saving to float16 or MXFP4 for VLLM\n",
        "\n",
        "We also support saving to `float16` directly. Select `merged_16bit` for float16 or `mxfp4` for MXFP4 (OpenAI's GPT-OSS native precision). We also allow `lora` adapters as a fallback. Use `push_to_hub_merged` to upload to your Hugging Face account! You can go to https://huggingface.co/settings/tokens for your personal tokens."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "NjXGTkp7YNtB"
      },
      "outputs": [],
      "source": [
        "# Merge and push to hub in mxfp4 4bit format\n",
        "if False:\n",
        "    model.save_pretrained_merged(\"finetuned_model\", tokenizer, save_method = \"mxfp4\")\n",
        "if False: model.push_to_hub_merged(\"repo_id/repo_name\", tokenizer, token = \"hf...\", save_method = \"mxfp4\")\n",
        "\n",
        "# Merge and push to hub in 16bit\n",
        "if False:\n",
        "    model.save_pretrained_merged(\"finetuned_model\", tokenizer, save_method = \"merged_16bit\")\n",
        "if False: # Pushing to HF Hub\n",
        "    model.push_to_hub_merged(\"hf/gpt-oss-finetune\", tokenizer, save_method = \"merged_16bit\", token = \"\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V15Yhj1V9lwG"
      },
      "source": [
        "And we're done! If you have any questions on Unsloth, we have a [Discord](https://discord.gg/unsloth) channel! If you find any bugs or want to keep updated with the latest LLM stuff, or need help, join projects etc, feel free to join our Discord!\n",
        "\n",
        "Some other links:\n",
        "1. Train your own reasoning model - Llama GRPO notebook [Free Colab](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Llama3.1_(8B)-GRPO.ipynb)\n",
        "2. Saving finetunes to Ollama. [Free notebook](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Llama3_(8B)-Ollama.ipynb)\n",
        "3. Llama 3.2 Vision finetuning - Radiography use case. [Free Colab](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Llama3.2_(11B)-Vision.ipynb)\n",
        "6. See notebooks for DPO, ORPO, Continued pretraining, conversational finetuning and more on our [documentation](https://docs.unsloth.ai/get-started/unsloth-notebooks)!\n",
        "\n",
        "<div class=\"align-center\">\n",
        "  <a href=\"https://unsloth.ai\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png\" width=\"115\"></a>\n",
        "  <a href=\"https://discord.gg/unsloth\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/Discord.png\" width=\"145\"></a>\n",
        "  <a href=\"https://docs.unsloth.ai/\"><img src=\"https://github.com/unslothai/unsloth/blob/main/images/documentation%20green%20button.png?raw=true\" width=\"125\"></a>\n",
        "\n",
        "  Join Discord if you need help + ⭐️ <i>Star us on <a href=\"https://github.com/unslothai/unsloth\">Github</a> </i> ⭐️\n",
        "</div>\n"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "gpuType": "T4",
      "include_colab_link": true,
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "0212e4e49d894c43853e242dc0ff075b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_7c49c46974c84d03bace5136eaff00a0",
              "IPY_MODEL_2e1aaed287bd4fe1b79b89351d061974",
              "IPY_MODEL_8c2431afcf0e46c8819c4e179131b141"
            ],
            "layout": "IPY_MODEL_d65fa96d223247ee81e31cfd7ac4f8da"
          }
        },
        "03b9552ca9ae4621ba2815f908b975d1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a939624297804d0bad54b0773e0db3f4",
            "placeholder": "​",
            "style": "IPY_MODEL_6f15f8571f76440c895e46729cdf2aa3",
            "value": " 22.8k/? [00:00&lt;00:00, 1.77MB/s]"
          }
        },
        "0ab594483aea4f89baade12417c6e660": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0c832012a6144427991f497daf31f4b4": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0f8413eb46174743bfbe830688060810": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "10295e2b43664d739cf081bf522df28c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2840a2cdc19c4b76a9c42b60598ace93",
            "placeholder": "​",
            "style": "IPY_MODEL_edbe3470295a4afa911fc530ea9fcd39",
            "value": " 27.9M/27.9M [00:00&lt;00:00, 53.4MB/s]"
          }
        },
        "158a8752ca2c431591ca86a2429a6af1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_0ab594483aea4f89baade12417c6e660",
            "placeholder": "​",
            "style": "IPY_MODEL_0f8413eb46174743bfbe830688060810",
            "value": " 446/446 [00:00&lt;00:00, 41.8kB/s]"
          }
        },
        "16523d281b774262bce76f78ebd63c38": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "17147264eb4f45b49dc9c7a5679ef999": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d7b6b0f2054e4fc58461f33e1932f85f",
            "max": 165,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_c2b69f452163466caa3e0429c16d3779",
            "value": 165
          }
        },
        "19d6b280f3104bd793e7127d88de072f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9b0bdefedcdb4564b4dd658be51405c0",
            "max": 4,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_b2788808eaab4575b3cb2240637faef2",
            "value": 4
          }
        },
        "1a994b74c9b545789f2323501d2d2d13": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1e59695a79d5432595ae7e7656414435": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1fd1a5f6611544989ef7754549f3860c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "2433e937a96f40409e0e7193978fe652": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_29aa7f1afd01405d9d4de805348bd096",
              "IPY_MODEL_f47d970082e1470ab8704b272acba841",
              "IPY_MODEL_03b9552ca9ae4621ba2815f908b975d1"
            ],
            "layout": "IPY_MODEL_610d85d07c444ed5a4dc2730ef5f471d"
          }
        },
        "282a6d45569440d6918c62932f1eec10": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2840a2cdc19c4b76a9c42b60598ace93": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "28bb87ab20974ab6a4729006cc3bbaaf": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "29aa7f1afd01405d9d4de805348bd096": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_75e33b5b822142f9b44e0ab50930c56e",
            "placeholder": "​",
            "style": "IPY_MODEL_8d67ff84c84f4a4f8ff45b34b992e6e1",
            "value": "tokenizer_config.json: "
          }
        },
        "2c43ef681a71479ca11f61052ae1b631": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2c9c502defb647819099a26c48600435": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2e1aaed287bd4fe1b79b89351d061974": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5d2c63aa3eeb43088233458fc2fa003b",
            "max": 1,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_9f2b3ad4c44f457abd5dc69b5f2e7c79",
            "value": 1
          }
        },
        "2f449055f16046b2be805fa66c6ac1ad": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": "20px"
          }
        },
        "2facb22996e34da489b7b6cde4856b2f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3159a22f1d9542c5850ba3d9a6aa833d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_7e17a72fc6414ea8aa12824dc00d79be",
              "IPY_MODEL_fe21b55cf55f43f59fa47fc40f9d13b1",
              "IPY_MODEL_4a0a2d1e7337430e83ae3f70cc602f63"
            ],
            "layout": "IPY_MODEL_0c832012a6144427991f497daf31f4b4"
          }
        },
        "3179634b83e24ede8271df0c5ea4bb7f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a03ed89c12ba451c8d5ac5d873c289fa",
            "placeholder": "​",
            "style": "IPY_MODEL_b3ba05b1e0e84c188011a81a30ce2308",
            "value": " 4.00G/4.00G [01:18&lt;00:00, 54.6MB/s]"
          }
        },
        "34ef05f077114be1992a3148f640e74e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "3d74a20355de461bbc0585ddd4320a8a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3e14a3169c4448149571bdbdb56f7dfd": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3f61649c340d49c6974314be997117a0": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "40eec74c298341c9a85fb3da301bf13c": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "426912b358c14c3995dbed98443892b6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_a4969d5533bf4435accf29b9925874ba",
              "IPY_MODEL_d9c6e9f988bd4db78f128b1c97833c0f",
              "IPY_MODEL_9e8fc61c9cca43318cf9aa7d4089eccb"
            ],
            "layout": "IPY_MODEL_c912ff0c768648b6a49e3e0e5d5efc90"
          }
        },
        "4a0a2d1e7337430e83ae3f70cc602f63": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_76c4f3870855405f9ab2be61fc7aac46",
            "placeholder": "​",
            "style": "IPY_MODEL_63721834805341adabb9cb4818cd3879",
            "value": " 3.37G/3.37G [01:18&lt;00:00, 11.0MB/s]"
          }
        },
        "4bfeabbb9ec7483fa978e07abd67ad4a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "4ea27b87d18d4511842e6ba0c5df8378": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "4ee55509ddbc4bd88a6809cb9a549c1d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "4fbbd316eaac46cabbe6d0b185a86095": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_28bb87ab20974ab6a4729006cc3bbaaf",
            "placeholder": "​",
            "style": "IPY_MODEL_bc195d30f2e5474f8c8aef2d4c09c6f9",
            "value": "tokenizer.json: 100%"
          }
        },
        "58e9e79ea73d4bb3a719e52a3d7b1b93": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5caf1410508b47cc9725e6f7947448d0": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "5d2c63aa3eeb43088233458fc2fa003b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": "20px"
          }
        },
        "609f889e0e8a4d6880f337b7ba7078d2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2c9c502defb647819099a26c48600435",
            "placeholder": "​",
            "style": "IPY_MODEL_5caf1410508b47cc9725e6f7947448d0",
            "value": "generation_config.json: 100%"
          }
        },
        "60f337003c6f49c8beb302d537528a7b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_282a6d45569440d6918c62932f1eec10",
            "max": 27868174,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_841a71f1893a44b2bf7c9d3604187e2c",
            "value": 27868174
          }
        },
        "610d85d07c444ed5a4dc2730ef5f471d": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "627483e9119d4cf691d5eb1c3b043e94": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "63721834805341adabb9cb4818cd3879": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "64fb75c422d14aed9bfc5b028fe2544e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "66d580aee78c4fd585b13cf7f19975bb": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "699c41a68613456f9580541e8ffba701": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_994d8a5a8af14aa7a57283f05af273e3",
            "placeholder": "​",
            "style": "IPY_MODEL_cd856add8a7e43f8a6fafeae01c518d5",
            "value": "Loading checkpoint shards: 100%"
          }
        },
        "6de18860224a4d3e8145be34e9fc886e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1e59695a79d5432595ae7e7656414435",
            "max": 446,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_fb0f2517dca047a19a3dc2a81c620cd3",
            "value": 446
          }
        },
        "6f15f8571f76440c895e46729cdf2aa3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "73d67c3d579c4e33913905cc9f30caf8": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "73ede82185b7479eb040a66dcd4e723b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a1d08182271540d8947dc83b633e13f2",
            "placeholder": "​",
            "style": "IPY_MODEL_66d580aee78c4fd585b13cf7f19975bb",
            "value": " 165/165 [00:00&lt;00:00, 9.84kB/s]"
          }
        },
        "747b22b5a7044fba9106e638f88d9c52": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "750d058daf63451e87a81e39fe15e0c8": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_609f889e0e8a4d6880f337b7ba7078d2",
              "IPY_MODEL_17147264eb4f45b49dc9c7a5679ef999",
              "IPY_MODEL_73ede82185b7479eb040a66dcd4e723b"
            ],
            "layout": "IPY_MODEL_2c43ef681a71479ca11f61052ae1b631"
          }
        },
        "75e33b5b822142f9b44e0ab50930c56e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "76c4f3870855405f9ab2be61fc7aac46": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "791dc643382446c68d18727d5d7e76dc": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_9d9c1c27871a427384fe7abacc1d7863",
              "IPY_MODEL_6de18860224a4d3e8145be34e9fc886e",
              "IPY_MODEL_158a8752ca2c431591ca86a2429a6af1"
            ],
            "layout": "IPY_MODEL_cc7c1b8e16ad4384a8b7239d0ed9f781"
          }
        },
        "7c49c46974c84d03bace5136eaff00a0": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_16523d281b774262bce76f78ebd63c38",
            "placeholder": "​",
            "style": "IPY_MODEL_ff4bd1c9fd2840718474237981fb55f5",
            "value": "model.safetensors.index.json: "
          }
        },
        "7db7a817c64540fd91e395db7ce381cf": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ad3c48ed896f49b1a2b544577d271f9e",
            "max": 3998751275,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_fdc97d76ff604536822e7c38feab0a95",
            "value": 3998751275
          }
        },
        "7e17a72fc6414ea8aa12824dc00d79be": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3f61649c340d49c6974314be997117a0",
            "placeholder": "​",
            "style": "IPY_MODEL_bce6bd44eac94c4c8b706a1b54c9f467",
            "value": "model-00003-of-00004.safetensors: 100%"
          }
        },
        "7e758da3adcd47578a3a74f74b7012f8": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7eced28c016b47b29bf6a0feca16b559": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7f70ee5fe26542cdb3066c3eba1fbf52": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_9ca972f14a554a8bac55b453e17ab783",
              "IPY_MODEL_bdcec5febe9940febd4cb093bfb6ff76",
              "IPY_MODEL_fcafc324e1314662a955823de312fac2"
            ],
            "layout": "IPY_MODEL_7eced28c016b47b29bf6a0feca16b559"
          }
        },
        "841a71f1893a44b2bf7c9d3604187e2c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "87fa027dfa2744e894af20b0d567a0b9": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "89e3ca07e6ac4fb39a73013ae8b03d1d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "8c2431afcf0e46c8819c4e179131b141": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_bb94fe6e61c041e093335c961a803a02",
            "placeholder": "​",
            "style": "IPY_MODEL_9b6578ac49124239950d34d4522e0370",
            "value": " 1.19M/? [00:00&lt;00:00, 79.8MB/s]"
          }
        },
        "8d67ff84c84f4a4f8ff45b34b992e6e1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "95023494e71146bfa17b19246cc95e14": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1a994b74c9b545789f2323501d2d2d13",
            "placeholder": "​",
            "style": "IPY_MODEL_3d74a20355de461bbc0585ddd4320a8a",
            "value": "model-00001-of-00004.safetensors: 100%"
          }
        },
        "994d8a5a8af14aa7a57283f05af273e3": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9b0bdefedcdb4564b4dd658be51405c0": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9b6578ac49124239950d34d4522e0370": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9c2b813e759745f19cde07fe7a58aafd": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9ca972f14a554a8bac55b453e17ab783": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d03ebf442e57459fbe256027d9f69bf3",
            "placeholder": "​",
            "style": "IPY_MODEL_abb9337fe1f3452697d9d3acea9c9257",
            "value": "chat_template.jinja: "
          }
        },
        "9d9c1c27871a427384fe7abacc1d7863": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9c2b813e759745f19cde07fe7a58aafd",
            "placeholder": "​",
            "style": "IPY_MODEL_d846d0fb4a5e45ff97162065438ec095",
            "value": "special_tokens_map.json: 100%"
          }
        },
        "9e8fc61c9cca43318cf9aa7d4089eccb": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d44c37caf8e4471395f7b73cc6c00c5b",
            "placeholder": "​",
            "style": "IPY_MODEL_627483e9119d4cf691d5eb1c3b043e94",
            "value": " 4.00G/4.00G [01:17&lt;00:00, 65.9MB/s]"
          }
        },
        "9ec47ccd1b5c4f109b903fe7117aee4c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_4fbbd316eaac46cabbe6d0b185a86095",
              "IPY_MODEL_60f337003c6f49c8beb302d537528a7b",
              "IPY_MODEL_10295e2b43664d739cf081bf522df28c"
            ],
            "layout": "IPY_MODEL_f280fb5f5cec4a929b18e0bab769a7a8"
          }
        },
        "9f2b3ad4c44f457abd5dc69b5f2e7c79": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "a03ed89c12ba451c8d5ac5d873c289fa": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a1d08182271540d8947dc83b633e13f2": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a4969d5533bf4435accf29b9925874ba": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_da01935bfbb44c91b585a88994513905",
            "placeholder": "​",
            "style": "IPY_MODEL_3e14a3169c4448149571bdbdb56f7dfd",
            "value": "model-00002-of-00004.safetensors: 100%"
          }
        },
        "a939624297804d0bad54b0773e0db3f4": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "aa1434fd586341c398233f7a3aa14a55": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "abb9337fe1f3452697d9d3acea9c9257": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "ad0f3014365149d8a3846c1725193847": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": "20px"
          }
        },
        "ad3c48ed896f49b1a2b544577d271f9e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ad5f82f0f56a44f5a9a7d5bd4fd1bf0f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_d4e66ae0819b48cdbe9983218029dd35",
              "IPY_MODEL_ddb4f938a1cf4871ab9a90382a97d6ee",
              "IPY_MODEL_b54e55595df94453af8d41ba9cb1dc69"
            ],
            "layout": "IPY_MODEL_40eec74c298341c9a85fb3da301bf13c"
          }
        },
        "b2788808eaab4575b3cb2240637faef2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "b3ba05b1e0e84c188011a81a30ce2308": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "b54e55595df94453af8d41ba9cb1dc69": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_58e9e79ea73d4bb3a719e52a3d7b1b93",
            "placeholder": "​",
            "style": "IPY_MODEL_4ea27b87d18d4511842e6ba0c5df8378",
            "value": " 1.16G/1.16G [00:30&lt;00:00, 50.3MB/s]"
          }
        },
        "b5c08cff1905495abf995753db788a0c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_dfd98f5746074f4b83c1ff1ce414c10b",
            "max": 4,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_b5ecf6db72654682b5f4fd984a918a47",
            "value": 4
          }
        },
        "b5ecf6db72654682b5f4fd984a918a47": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "b85a957353094460b4d87dfc4404d731": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "bb94fe6e61c041e093335c961a803a02": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "bbe67c649fdb4cc4b9067d7106b48952": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_fb7d044d8f434242affb7d5027312beb",
            "placeholder": "​",
            "style": "IPY_MODEL_e293d0061a3842dbb4ad615128105638",
            "value": " 4/4 [00:55&lt;00:00, 11.99s/it]"
          }
        },
        "bc0b6f73b0174e87a7b1fabfc02ac693": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "bc195d30f2e5474f8c8aef2d4c09c6f9": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "bce6bd44eac94c4c8b706a1b54c9f467": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "bd80351aa7cc497b901d8264cb89d6fd": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "bdcec5febe9940febd4cb093bfb6ff76": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ad0f3014365149d8a3846c1725193847",
            "max": 1,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_1fd1a5f6611544989ef7754549f3860c",
            "value": 1
          }
        },
        "c199d1e74ff44dc3a1bae8b1f3f3e5a0": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_699c41a68613456f9580541e8ffba701",
              "IPY_MODEL_b5c08cff1905495abf995753db788a0c",
              "IPY_MODEL_bbe67c649fdb4cc4b9067d7106b48952"
            ],
            "layout": "IPY_MODEL_bc0b6f73b0174e87a7b1fabfc02ac693"
          }
        },
        "c24955eee4f942abb5806e04ff209b05": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c2b69f452163466caa3e0429c16d3779": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "c44dfe5323024921a8d8120c7863c24b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_73d67c3d579c4e33913905cc9f30caf8",
            "placeholder": "​",
            "style": "IPY_MODEL_bd80351aa7cc497b901d8264cb89d6fd",
            "value": "Fetching 4 files: 100%"
          }
        },
        "c912ff0c768648b6a49e3e0e5d5efc90": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c9aca6022ac242db8228cbdba9446b6b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "cc7c1b8e16ad4384a8b7239d0ed9f781": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "cd856add8a7e43f8a6fafeae01c518d5": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d03ebf442e57459fbe256027d9f69bf3": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d44c37caf8e4471395f7b73cc6c00c5b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d4e66ae0819b48cdbe9983218029dd35": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b85a957353094460b4d87dfc4404d731",
            "placeholder": "​",
            "style": "IPY_MODEL_4bfeabbb9ec7483fa978e07abd67ad4a",
            "value": "model-00004-of-00004.safetensors: 100%"
          }
        },
        "d5e6d751713d4f3f8a0ff6bc934fa68e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_95023494e71146bfa17b19246cc95e14",
              "IPY_MODEL_7db7a817c64540fd91e395db7ce381cf",
              "IPY_MODEL_3179634b83e24ede8271df0c5ea4bb7f"
            ],
            "layout": "IPY_MODEL_e5b5fe76a7134f95a5ee0b2b736f0ff2"
          }
        },
        "d65fa96d223247ee81e31cfd7ac4f8da": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d7b6b0f2054e4fc58461f33e1932f85f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d846d0fb4a5e45ff97162065438ec095": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d9c6e9f988bd4db78f128b1c97833c0f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2facb22996e34da489b7b6cde4856b2f",
            "max": 3996690997,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_34ef05f077114be1992a3148f640e74e",
            "value": 3996690997
          }
        },
        "da01935bfbb44c91b585a88994513905": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "daee7532f84d4295b8b7faca729bc156": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "ddb4f938a1cf4871ab9a90382a97d6ee": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c9aca6022ac242db8228cbdba9446b6b",
            "max": 1158267008,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_89e3ca07e6ac4fb39a73013ae8b03d1d",
            "value": 1158267008
          }
        },
        "dfd98f5746074f4b83c1ff1ce414c10b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e050c7afa4124671842cd52cfa86a973": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_c44dfe5323024921a8d8120c7863c24b",
              "IPY_MODEL_19d6b280f3104bd793e7127d88de072f",
              "IPY_MODEL_f4c9a36fc6bd47e48c48a3433b6657e0"
            ],
            "layout": "IPY_MODEL_aa1434fd586341c398233f7a3aa14a55"
          }
        },
        "e293d0061a3842dbb4ad615128105638": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e5b5fe76a7134f95a5ee0b2b736f0ff2": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "edbe3470295a4afa911fc530ea9fcd39": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f280fb5f5cec4a929b18e0bab769a7a8": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f47d970082e1470ab8704b272acba841": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2f449055f16046b2be805fa66c6ac1ad",
            "max": 1,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_daee7532f84d4295b8b7faca729bc156",
            "value": 1
          }
        },
        "f4c9a36fc6bd47e48c48a3433b6657e0": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_87fa027dfa2744e894af20b0d567a0b9",
            "placeholder": "​",
            "style": "IPY_MODEL_747b22b5a7044fba9106e638f88d9c52",
            "value": " 4/4 [01:19&lt;00:00, 20.50s/it]"
          }
        },
        "fb0f2517dca047a19a3dc2a81c620cd3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "fb7d044d8f434242affb7d5027312beb": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "fcafc324e1314662a955823de312fac2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_7e758da3adcd47578a3a74f74b7012f8",
            "placeholder": "​",
            "style": "IPY_MODEL_64fb75c422d14aed9bfc5b028fe2544e",
            "value": " 15.1k/? [00:00&lt;00:00, 1.42MB/s]"
          }
        },
        "fdc97d76ff604536822e7c38feab0a95": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "fe21b55cf55f43f59fa47fc40f9d13b1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c24955eee4f942abb5806e04ff209b05",
            "max": 3372033380,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_4ee55509ddbc4bd88a6809cb9a549c1d",
            "value": 3372033380
          }
        },
        "ff4bd1c9fd2840718474237981fb55f5": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "state": {}
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
